Real time brachytherapy spatial registration and visualization system

ABSTRACT

A method and apparatus for three-dimensional imaging and treatment of a patient&#39;s body. The method and apparatus utilize a system for developing a therapy plan for treatment of an organ of the patient, a device for generating ultrasound image data from a treatment region and a device for providing a translucent volume image of a portion of a patient&#39;s body and a separate translucent image of the patient organ and a three dimensional viewing device to superimpose a translucent article image to enable viewing of the article image simultaneously with the patient organ and a portion of the patient&#39;s body.

This invention is a continuation-in-part of U.S. application Ser. No.08/977,362, filed on Nov. 24, 1997.

As part of this specification a microfiche appendix has been preparedwith one page of fiche having a total of 78 frames, including the testtarget frame.

The present invention is directed in general to an improved method andapparatus for carrying out minimally invasive treatments of the humanbody by virtual reality visualization of the treatment area. Moreparticularly the invention is concerned with use of an apparatus andmethod for providing real time images of a human anatomy undergoingtreatment along with rapid radiation seed therapy planning and rapidperformance of therapy including an automatic seed loading methodologywhich enhances therapeutic treatment with greatly improved efficiencyboth in terms of time and resources.

New minimally invasive surgical procedures are most often opticallyguided, but such optical guidance methods do not permit visualizationand guidance of instruments or probes within (inside) the target tissueor organ. Incorporation of real-time three-dimensional visualizationinside diseased tissues would provide accurate guidance of therapy.Open-magnet MRI is used to visualize some procedures such as thermaltherapy and brain biopsies. However, the method is expensive, not trulyreal-time, and is limited in application.

Numerous conventional treatment methods involve attempts to provide atargeted dosage of radiation or chemicals to the organ, and suchtreatments are often based on general anatomical assumptions of size andlocation. These methods suffer from inaccuracy of localizing the targetfor any one particular individual and potential real time changes ofrelative orientation and position of target tissue, normal tissue, andradiation therapy devices.

It is instructive in explaining the invention to consider one specifictype of exemplary condition, adenocarcinoma of the male prostate whichis the most commonly diagnosed cancer in the male population of theUnited States. At present, 254,000 new cases of prostate cancer werediagnosed in 1995 and 317,000 in 1996. In the 1960s, a method ofimplanting radioactive gold or iodine seeds was developed. With thisapproach, the radioactive material is permanently placed into theprostate via a retropubic approach during laparotomy when diagnosticlymphadenectomy was also being performed. A high dose of radiation isdelivered to the prostate as the radioactive seeds decay. In severalreports, the five year disease free survival ("local control") obtainedby this method was compared to similarly staged patients treated with anexternal radiation beam. In view of this, gold was replaced by I¹²⁵implantation for safety of personnel doing implantation. Except forearly stage prostate cancer (T2a tumors), inferior rates of localcontrol are reported with "free hand" 125-Iodine implantation. There wassignificant dose inhomogeneity due to the nonuniformity of seedplacement, leading to underdosing of portions of the prostate gland andsignificant complications due to overdosing of adjacent healthy tissuestructures. The poor results for local control and normal tissuecomplication were attributed to the doctor's inability to visualize andhence control where the radioactive seeds were actually being depositedinside the patient.

Recently, transrectal ultrasonography ("TRUS") has been used tovisualize 125-Iodine seed placement during transperineal implantation.The early reported rates of serious late complications is higher thanexternal beam therapy. Even with this technique, significantimprecisions in seed placement are observed. Due to the proximity of theprostate to the rectum and bladder, incorrect seed placement may lead toserious overdosing of these structures and late complications.

The recent transrectal ultrasound guided transperineal implant techniquehas been developed which is in use. That procedure is described in threesteps: (1) the initial volumetric assessment of the prostate glandperformed using ultrasound, (2) development of a radiation therapy"pre-plan," and (3) performing the actual intraoperative implant. Thepurpose of the initial volumetric assessment prior to the pre-plan orimplantation is to obtain a quantitative understanding of the size ofthe prostate, which is then used to determine the total activity anddistribution of radioactivity which is to be implanted into theprostate. To perform the assessment, an ultrasound probe is physicallyattached to a template. The template is a plastic rectangle whichcontains an array of holes separated at predefined intervals, usually 5mm. The template system serves two purposes: (1) to fix the ultrasoundprobe, and hence the imaging plane to the reference frame of thecatheter and seed positions, and (2) to guide the catheters into theprostate volume. More specifically, the template system serves as areference frame for spatial quantities which are required for thedescription of the implant procedure. Using transrectal ultrasound, anumber of serial ultrasound images are obtained at 5-mm intervals, andthe prostate is outlined on each image. The images are taken so that theentire prostate gland is covered. This results in a stack oftwo-dimensional outlines, or contours, which, taken together, outlinethe entire three-dimensional prostate volume. From this volume, thequantitative volume of the prostate is calculated.

Once the three-dimensional contour data has been obtained for theprostate volume, a radiation therapy plan which describes the positionsof the radioactive seeds within the prostate is developed. This planattempts to optimize the dose to the prostate, minimize the dose tosurrounding healthy tissue, and minimize dose inhomogeneity. Thepositions of the radioactive seeds are constrained to fall within thecatheter tracks, since the seeds are placed within the prostatetransperineally via these catheters. The result of the pre-plandescribes the positions and strengths of the radioactive seeds withinthe catheter which optimizes the dose to the prostate.

Intraoperatively, the TRUS probe is inserted, and the template ismounted against the perineum. As previously described, the template is aplastic rectangle which contains an array of holes separated at fixedintervals. These holes act as guides for the catheters. The TRUS probeis inserted into the rectum and placed so that the image corresponds tothe prostate base (the maximum depth). Two or three catheters areinserted into the tissue surrounding the prostate or in the periphery ofthe prostate to immobilize the gland. These catheters contain noradioactive seeds. This image serves as a spatial reference for allfurther images and seed positions within the prostate. Subsequently,catheters are inserted into the gland based on the pre-plan through thetemplate. The ultrasound probe is positioned each time so that thecatheter, and hence seeds, which are inserted into the prostate arevisible on the ultrasound image. If the placement of the catheter withinthe prostate is not according to the pre-plan, the catheter is thenwithdrawn and reinserted until the catheter is correctly placed. This isa time-consuming process; and it is very difficult to achieve optimalplacement. Invariably, the catheters deflect angularly as they areinserted, and their positions are difficult to determine bytwo-dimensional ultrasound. This is due to the fact that thevisualization process is a two-dimensional process while the actualimplant procedure is three-dimensional. Once all the seeds are in place,another series of two-dimensional images are obtained to quantify thefinal, resultant dose distribution delivered to the patient. In someinstances, a pair of orthogonal fluoroscopic images are also obtained todetermine the final seed placements. This procedure is usually performeda few weeks post implant.

These above described prior art systems suffer from inherent inaccuracy,the inability to correct the positioning of the radioactive seedswithout repeated withdrawal and reinsertion of seeds into the prostateand are not real time manipulations of the therapeutic medium. Further,the overall positioning of the template and patient may be differentduring treatment compared to the assessment phase. Consequently, thecatheter position and seed position may be at an undesired positionrelative to the presumed assessment phase location.

It is therefore an object of the invention to provide an improved systemand method for invasive treatment of the human body.

It is another object of the invention to provide a novel system andmethod for real time and/or near real time, three-dimensionalvisualization of a human organ undergoing invasive treatment.

It is also an object of the present invention to provide a more preciseand accurate implant placement for radiation therapy, thermal therapy,and surgical ablation.

It is also an object of the invention to provide an improved system andmethod for generating a three-dimensional image data set of a humanorgan for a treatment protocol using a real-time ultrasound imagingsystem with spatial landmarks to relate the image data set to presenttime, invasive treatment devices.

It is a further object of the invention to provide a novel system andmethod for spatial registration of two-dimensional and three-dimensionalimages of a human organ, such as the human prostate, with the actuallocation of the organ in the body.

It is an additional object of the invention to provide an improvedmethod and system for three-dimensional virtual imaging of the maleprostate gland and overlaid virtual imaging of devices being insertedinto the prostate for deposition of radioactive seeds for cancertherapy.

It is yet a further object of the invention to provide an automatedmethod and system for loading of radioactive therapeutic treatment seedsbased on a clinical plan enabling rapid treatment based on substantiallyreal time pre-planning using rapid patient organ evaluation.

These and other objects and advantages of the invention will be readilyapparent from the following description of the preferred embodimentsthereof, taken in conjunction with the accompanying drawings describedbelow.

BRIEF DESCRIPTION OF THE DRAWINGS

The file of this patent contains at least one drawing executed in color.Copies of this patent with color drawings will be provided by the Patentand Trademark Office upon request and payment of the necessary fee.

FIG. 1A illustrates a block diagram of an embodiment of the inventionand FIG. 1B shows an alternate embodiment for a three-dimensional probe;

FIG. 2 illustrates an ultrasound guided implant system;

FIG. 3A illustrates patient setup for a radioactive implant procedure;FIG. 3B illustrates an anatomical prostate phantom used for testing andplanning; and FIG. 3C illustrates in detail a probe holder/stepperassembly shown partly in FIG. 3A;

FIG. 4A illustrates a front schematic view of a brachytherapy phantomand FIG. 4B a side schematic view of the brachytherapy phantom;

FIG. 5A illustrates reconstruction of standard orthogonal image planesfrom a three-dimensional image stack and FIG. 5B the reconstruction ofoblique image planes from a three-dimensional image stack;

FIG. 6 illustrates the viewing geometry for a three-dimensionaltranslucent reconstruction of an image;

FIG. 7A illustrates translucent images of a human prostate for fourdifferent viewing angles and FIG. 7B illustrates translucent images of aphantom organ for six different viewing angles;

FIG. 8 illustrates a time sequenced image of the prostate organ in FIG.7A showing approach of a catheter containing a radioactive seed,deposition of the seed and withdrawal of the catheter leaving the seed;

FIG. 9 illustrates isodose distributions of radiation from a singleradioactive seed;

FIG. 10 illustrates a flow chart of software routine for processingimaging data for visualization;

FIG. 11 illustrates a virtual reality head mounted display;

FIG. 12 illustrates a flow diagram of software module operativeconnections;

FIG. 13A illustrates a perspective view of a stepper assembly with theprobe in position and FIG. 13B illustrates a perspective view of theprobe stepper along with a probe stabilization system; and

FIG. 14 illustrates a redundant monitoring and automatic loading systemfor radioactive seeds and inert spacers.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

A system 10 constructed in accordance with an example of the inventionis illustrated generally in FIG. 1. A three-dimensional probe 12accumulates image data from a treatment region or organ of a patient,image data is processed using a three-dimensional imaging card 14. Theprobe 12 preferably is an ultrasound device but can be any other rapidimaging technology, such as rapid CT or MR. A conventional personalcomputer 16 having a monitor can be used to operate on the image datafrom the imaging card 14 using conventional software and hardware toolsto be described in more detail hereinafter. Radioactive seeds 18 areprovided for insertion using any one of a variety of conventional meansfor inserting devices or articles into the human body, such as insertiondevices 19, which may be either needles or stiff catheters. Thethree-dimensional ultrasound probe 12, therefore, provides an imagesignal to the computer 16 and a virtual realty interface card 13 coupledto the imaging card 14 which enables a user to visualize a translucentimage of the patient organ and real time interaction of any one of avariety of treatment devices, such as the implant needles 19 or a Foleycatheter 20, and one of the seeds 18 within the organ. Computer softwarecan be utilized in a conventional manner to visualize thethree-dimensional imaging data in various formats (see the microficheAppendix and discussion hereinafter). The formats include orthogonal twodimensional images, oblique two-dimensional images, and translucentthree-dimensional rendering. All of these reconstructions can bedirectly displayed on the computer monitor; and three-dimensionaltranslucent, stereoscopic, rendering is also available in the VR(Virtual Realty) mode.

One of the preferred ultrasound probe 12 for example, is a conventionalKretz ultrasound imaging system manufactured by Kretz Corporation, nowavailable as Medison Combison 530 through Medison America Corporation,Pleasantown, Calif. This system and other such conventional systems arereadily available and can provide real time ultrasound image data. TheMedison Combison ultrasound system incorporates an endorectal probewhich acquires multiple image planes in real time and in certainembodiments the software (see the microfiche Appendix) reconstructs thetranslucent three-dimensional volume. Another example is of a B&KLeopard ultrasound imaging system with endorectal imaging probe (Boston,Mass.). Alternate systems include biplanar two-dimensional imagingsystems with the probe mounted in a stepper motor driven holder forrapid automatic acquisition of multiple image planes.

In a most preferred form of the invention, the system 10 includescomputer software for real-time image acquisition, image contouring,dose calculation and display software, dose volume histograms,three-dimensional dose contours, post-implant seed localization, and thepatient scheduling spreadsheet software. The microfiche Appendix ofcomputer software shows how to implement these functionalities. FIG. 12illustrates the operative connection between modules of the software.The system software enables a two-dimensional and three-dimensionalimage visualization for brachytherapy employing two-dimensionalultrasound imaging for use in radioactive seed implants of the prostate.The software for the brachytherapy seed implant and dose calculationsystem was developed on a Pentium-based processor with supportinggraphics and digitizing hardware. The software consists oftwo-dimensional and three-dimensional routines. The two-dimensionaltools consist of standard imaging tools largely available for CT and MRIapplications. These tools include displays of the imaging volume in anyof the three standard orthogonal planes (transverse, sagittal, andcoronal), in addition to the ability to display the imaging in anyarbitrary, oblique imaging plane. Standard image processing tools suchas real time window leveling, zoom and pan will be available. Thethree-dimensional tools consist of a three-dimensional rendering of theactual contour slices imaging data. Based upon volumetric patientstudies, the prostate volume can be displayed. The user has the optionof viewing one or a mixture of two-dimensional and three-dimensionalsurface views on the monitor.

Contouring tools are also available for the user to draw with the mouseoutlines, or contours, of any structure visible on the imaging plane.Each contour can be varied as to color, line thickness, and line patternto aid in distinguishing between different contour sets.

Once a set of two-dimensional contours has been defied, either manuallyor automatically, on a number of different image slices they can bereconstructed in real time in the three-dimensional translucent view(described in more detail hereinafter). This results in a surfacerendering of the volume bounded by the contours. The surface renderingcan be chosen to be transparent, solid, or invisible (not rendered atall).

Once a seed has been placed into treatment position (details concerningseed implantation provided later), the user has the ability to displaythe dose of one or a set of seeds. The dose as a function of positionfor a cylindrical ¹²⁵ or ¹⁰³ Pd seed of a given activity can bedetermined from a lookup table or calculated from an analytic formula.The dose field can be visualized as a set of isodose lines intwo-dimensions or isodose surface in three-dimensions. The process ofconstructing an isodose line or surface is defined by simply drawing apoint for each pixel/voxel which contains a certain specified dosevalue. For example, the user can specify that the 137 Gy, 120 Gy, 100Gy, and 60 Gy isodose lines be drawn on the two-dimensional slice foreach image plane, and the 137 Gy isodose surface shown on thethree-dimensional rendered mode. Again, similar to the contouredvolumes, the isodose surface can be reconstructed in any of the userselected modes defined for contoured volumes.

The features/capabilities of the system software functionalitiesinclude: complete patient database archive and dose plan "playback";external image import capability; look-up tables for multiple seed kitsand template guides; multiple ultrasound imaging machine configurationcapability; image slice contouring using mouse, with edit capability;image cropping, image sizing, tiling, cascading; three-dimensionaldisplay of prostate, urethra, and other anatomies; rapid "on-line" dosecalculation in operating room/cysto suite during procedure; dose displaywith isodose lines, three-dimensional translucent, and ditheredisodoses; image export and printing (dose slices, contour slices, etc.);seed implant plan export and printing; dose volume histograms (withexport and printing); three-dimensional image support includingthree-dimensional image reconstruction from slices; three-dimensionaldisplay of isodose surfaces; image slice selection fromthree-dimensional image through any transverse plane; post-implantassessment including automatic seed localization; computer-controlledstepper; selection of manual (mouse entry), semi-automatic (buttonpush), or full automatic (computer-controlled stepper) ultrasound imagecollection.

For collecting ultrasound image data, the diagnostic transrectalultrasound probe 12 (see FIG. 2) is inserted into the patient's rectumto obtain real time volumetric images of the prostate for use during theimplant procedure. The diagnostic probe 12 is preferably a phased arrayprobe designed so that the array of transducers can rotate about theaxis of the array sweeping out a three-dimensional imaging volume. Asthe probe 12 rotates, images are captured and digitized by use of theimaging card 14 (see FIG. 1), so as to create a fixed number of imagesslices per rotation. An alternative method utilizes a transverseoriented phased array form of the endorectal probe 12 which is movedlongitudinally in an automated rapid sequence so as to create a seriesof transverse image slices automatically. Another embodiment of theprobe 12 can incorporate multiple transverse phased arrays (shown inphantom in FIG. 1B) arranged parallel to each other orthogonal to theaxis of an endorectal probe to produce multiple simultaneous imageslices (see, for example, FIGS. 5A and 5B). The three-dimensional imagedata will be represented as a three dimensional image raster.

The ultrasound probe 12 can be mounted into a probe holder 30 (see FIGS.3A and 3C) with FIG. 3B illustrating one example of an ultrasound imagefrom an anatomical prostate phantom employed to carry out testing andplanning. The probe holder 30 includes a digital encoder 42 forproviding information regarding the position of all of the desiredultrasound image planes in the prostate relative to each other. Theimage plane location will be automatically sent to the system computerand "tagged" to the acquired ultrasound image for that position (FIG.2). Thus, it will be possible to reproduce the longitudinal and lateralpositions of the implant catheters for the ultrasound therapyapplicators and for the temperature probes.

A probe holder/stepper assembly 21 (see FIG. 1A and in particular FIG.13) accommodates most ultrasound endorectal probes from variousmanufacturers. A "collett" 23 surrounds the probe 12 and is insertedinto the stepper/probe holder assembly 21. The stepper 21 is a digitaldevice with an automatic imaging link to the ultrasound machine and tothe remainder of the system 10. The stepper 21 has three digitallyencoded axes: main probe stage longitudinal axis 31, needle insertiontemplate longitudinal axis 33, and the rotational axis 35 of the imagingprobe itself. The stepper 21 automatically records the longitudinal(z-axis) position and sends that information to the computer 16.Whenever the user desires to acquire an image plane, the spatialposition of that image plane is automatically registered with thatimage. Thus, it requires less than a minute to digitally acquire anddocument all the image planes in a typical volume study. The stepper 21can be incrementally moved by the user with stepper knob 34 and thetemplate 25 can be stepped by template positioning control 37.

The holder/stepper assembly 21 can move the probe 12 in 2.5 mmincrements. A transrectal probe from B&K was used which operates at afrequency of 7.5 MHz and contains two sets of 128 transducer elementsforming both transverse and sagittal imaging assays. The imaging probe12 was moved via a knob on the side of the stepper 21 and its positionmeasured via a digitally interfaced optical position encoder. The probeholder/stepper 21 with transrectal probe 12 mounted is shown in FIG. 1.The real time multi-plane ultrasound probe 12 was modeled by obtainingsingle digitized transverse images at either 2.5 or 5 mm intervalsthrough the ultrasound prostate imaging phantom. The ultrasound prostatephantom is available from Computerized Imaging Reference Systems Inc.and contains a model of a prostate, urethra, and seminal vesiclesimmersed in a gel filled plastic box. The box has a cylindrical hole inthe base for the insertion and positioning of the transrectal probe anda perineal membrane for performing practice brachytherapy implants.FIGS. 4A and 4B display a schematic of the brachytherapy phantom. Oncethe static image slices have been digitized they were then inputted tothe software in a continuous cycle to model actual real time acquisitionof a full volume. Multiple sets of image slices can be obtained andrandomly cycled to more accurately simulate the actual three-dimensionalreal time ultrasound probe 12. The image slices are input to thesoftware transparently.

A probe stabilization system 27 (see FIG. 13B) is designed for use withany standard probe holder/stepper 21, yet it is optimized for use aspart of the system 10. This stabilization system 27 attaches easily andquickly to the cysto or operating room table using clamps 28, yetprovides maximum flexibility during patient setup. The stabilizationsystem 27 provides for five degrees of freedom of motion, yet is robustand stable. The probe stabilization system 27 includes a stepper probestand control 28 which allows up and down movement. Further motioncontrol is provided by stabilizer control 29 which enables up and downmotion and left to right along rods 30 (horizontal) and rods 31(vertical). Gross motions are positively controlled in a stable manner.Fine motions are obtained with the same controls and are exactlyreproducible.

A variety of the templates 25 (see FIG. 1) for the needles 19 can beused with the system 10. All of these implant templates are disposablepreferably. The system 10 can also accommodate use of other standardtemplates 25. The system software (see the microfiche Appendix) canstore the configuration of any number of the templates 25 for immediaterecall. Each template 25 stored in the system 10 is spatially registeredwith each ultrasound system configuration stored in the system software.

The system templates 25 provide assurance of sterility for patientcontact at a cost similar to that of sterilization of the usual standardtemplates. The disposable system templates 25 are a fraction of the costof standard reusable templates and provide greater safety.

There are several possible image processing cards which could beutilized; however, using current modalities each of the processing cardsis configured specifically for three-dimensional. The three-dimensionalimage raster is buffered; and thus, for example, if the two-dimensionalimages are 512×512 and there are sixteen image planes in the probe 12,and each pixel is a byte (256 gray scales), at least a 512×512×16byte=4.2 Mbyte image buffer in the card 14 is needed. Several commercialcards (for example, made by Coreco, Matrox and Integral Technologies)can be equipped with this amount of video RAM (VRAM), but the way thecard's hardware interacts with the computer's video and software driversdoes not utilize this data in three-dimensional. Current availablemethodologies enable augmenting the software and some hardware of thesecards so that they can act as a three-dimensional card. The processingand memory architecture preferably is designed to allow for simultaneousimage acquisition and processing. The digitizing card should alsopreferably have standard imaging tools, such as real time window andleveling, zoom and pan of the ultrasound images. Some existing cards(e.g., Matrox; Coreco) do provide standard imaging tools.

The three-dimensional image data arising from the ultrasound probe 12 ispreferably buffered on the imaging card 14. The three-dimensional imageis preferably represented as a series of two-dimensional images. This isreferred to as the image stack or three-dimensional image raster. Thethree-dimensional image raster is represented in memory as a lineararray of bytes of length N×M×P where N is the width of thetwo-dimensional image in pixels, M is the height a two-dimensional imagein pixels, and P is the number of two-dimensional images in the imagestack.

In a preferred embodiment the user can include defined formats. Entirethree-dimensional image stacks at specific times during theintraoperative session can be stored in the DICOM standard. The userwill have the ability to select a three-dimensional image volume forarchiving as part of the system software. These image stacks can then bereviewed in any of the various visualization modes (standard orthogonaltwo-dimensional views, oblique two-dimensional views, orthree-dimensional translucent views) as described above. In addition,the user will have the ability to store any of the two-dimensional viewsavailable at any time during the intraoperative session.

The computational platform can, for example, be any form of computingmeans, such as the personal computer 16, which incorporates a PCI busarchitecture. Currently, PCI bus is preferable over the ISA or EISA busbecause the PCI bus is much faster. However, a generic system which willbe suitable for this applicable will be described. A 200 MHz (or greaterspeed) Pentium/Pentium-Pro computer supplied with 128 Mbytes of RAM anda 6.0 Gbyte hard disk should be sufficient RAM and disk memory to runthe software in a real-time fashion and to archive all patient data.There should be sufficient RAM to facilitate host image processing inparallel with onboard image processing for quality assurance checks. Ahigh resolution monitor capable of displaying at least 1280×1024×64 bitresolutions is preferably used.

Based on currently available technology, the ultrasound images obtainedfrom the ultrasound imaging system of the ultrasound probe 12 can be ofgood diagnostic quality. When transforming this input image data into athree-dimensional representation, whether in the three-dimensionalperspective mode or the real time VR mode, the resultant volumes can,however, be noisy and hinder diagnostic and spatial accuracy. In orderto improve the image quality, a number of conventional hardware andsoftware filters can be used which will filter the incoming image datastored on the imaging card 14. Routines such as image pixel averaging,smoothing, and interpolation can improve the three-dimensional renderingof the imaging volume. These sets of filters or routines are to bedistinguished from the set of standard imaging tools running on the hostCPU which are available within a conventional imaging software package.

In the preferred embodiment, three of the perspective views are thestandard transverse, coronal and sagittal two-dimensional views. Thesethree orthogonal views are taken from a user specified location withinthe imaging space. For example, the user can request that the threeorthogonal views have their common centers at a spatial position of (5.0cm, 15.0, 25.0 cm) relative to the origin of the template system. Onealso can select the reference point of either of the three orthogonalviews independently, that is the three views do not have to have commoncenter points. As mentioned hereinbefore, FIGS. 5A and 5B show examplesof several example two-dimensional views from a three-dimensionalultrasound image volume. FIG. 6 shows a number of possible viewingdirections, and FIG. 7 gives further examples of translucentthree-dimensional viewing from different angles. The three-dimensionalultrasound image volume was obtained from actual ultrasound images of ahuman prostate and of a prostate implant phantom.

On each of the views, one can define, draw and edit contours usingconventional computer software, such as Microsoft Foundation Class (MFC)view files. Each contour can be given a unique name by the user, andthen drawn by the user using the mouse of the computer 16. Allattributes of the contours such as name and color can, based onconventional imaging software, be user selectable. The user can alsoedit the contours by selecting functions, such as adding a point to acontour, deleting a point from a contour or deleting the entire contour.Once the contours are defined, the user has the option to render them inthree-dimensional or view in conventional two-dimensional mode on thethree-dimensional perspective mode or viewed in the VR mode. Again, allcontour three-dimensional attributes such as color, lighting, andshading are user controlled. The contours by default appear on thetwo-dimensional images, however, the user can control the individualcontour's two-dimensional and three-dimensional visibility.

In order to improve the ability to visualize the real time,three-dimensional information, the three-dimensional image raster can berendered as a real time, transparent, three-dimensional volume. Thistransparent volume can be viewed and displayed on the monitor of thecomputer 16 at any arbitrary viewing angle and is calculated usingconventional three-dimensional object reconstruction algorithms. Suchstandard algorithms can render a large imaging volume in fractions of asecond, even on present day computing platforms. The transparent natureof the reconstruction thus allows the user to "see" inside any objectswhich appear in the imaging volume. For example, if the prostate isimaged in the imaging volume, then it will be reconstructed as atransparent volume, in which other anatomical landmarks such as theurethra, tissue abnormalities or calcifications can be seen. Inaddition, if any other objects such as needles or catheters are insertedinto the prostate, and if they are visible in the ultrasound images,they will be seen as they enter the prostate (see FIG. 8 showingintroduction of the seed 18 with the catheter/needle 19). Since thevolumes are rendered as transparent solids, the needles 19 (and otherarticles) can thus easily be seen as they move inside the prostatevolume as well. Since the ultrasound images are obtained in real time,the three-dimensional perspective reconstruction is also rendered inreal time. The preferred algorithm for the perspective three-dimensionalreconstruction is the known Bresenham ray-trace algorithm.

As described above, in the routine process of brachytherapy planning,the patient undergoes an initial volumetric ultrasound scan using theprobe 12. This scan is done before the radiation therapy planning or theactual implant. During the radiation therapy planning, the idealpositions of the radioactive seeds 18 (see FIG. 1) within the prostateare determined. This ideal seed distribution is optimized to deliver adose distribution within the prostate that will deliver all theradiation dose to the target volume only, while sparing the surroundinghealthy tissues such as the rectum and bladder. The optimal positions ofthe seeds 18 and the optimal position of the needles 19 are recorded forlater use in the operating room when the needles 19 are loaded into thepatient. The seeds 18 are then loaded into the needles 19, and thephysician then attempts to place the needles 19 inside the prostateusing a template 25 according to the treatment dose plan positions(again, see example in FIG. 8).

In the most preferred embodiment the seeds 18 are loaded through theneedles 19. A selection of different types of the seeds 18 (differentlevels of radioactivity) can be loaded through passageways, P, shown inFIG. 14. Optical sensors 90 and 91 are redundantly disposed adjacenteach of the passageways P with an associated microprocessor 93 and 97monitoring the number of the seeds 18 being instilled through the needle19. Radiation sensors 96 and 98 monitor the radiation activity of theseeds 18 being loaded into the needle 19. Spacers 100 are also instilledinto the needle 19 for separating the seeds 18 to achieve the desiredlevel of radiation activity and radiation contours. Optical sensors 92sense, redundantly as for the seeds 18, the passage of the spacers 100.

In a most preferred form of the invention, an automatic seed/needleloading method is implemented automatically loading implant needles 19with the radiation seeds 18 and spacers 29 based upon a pre-plan (doseplan) determined in the operating room (OR). This method accommodatesthe spacers 29 and separate leaded-acrylic see-through "bins" for theseeds 18 of two different activity levels. Thus, the needles 19 can beauto-loaded based upon optimal dose plans requiring seeds of differentactivity levels. The automatic seed/needle loading method and systeminterfaces directly to the computer 16 and reads the dose planinformation using the software of the microfiche Appendix. A display onthe auto-loader then displays to the operator each needle number,template coordinate location, and status of needle loading. Each of theneedles 19 are attached one at a time to the auto-loader assembly with astandard luer lock. The auto-loader has a sensor at the needleattachment point which detects if the needle 19 is attached for loading.Each of the needles 19 are then loaded in accordance with the pre-plan.

The automatic seed/needle loading method and system is thereforecompletely double-redundant, as mentioned hereinbefore. It incorporatesthe use of two totally independent microprocessors 93 and 94 whichconstantly check each other. Both the microprocessors 93 and 94 are alsoin communication with the system computer 16. The seeds 18 and thespacers 29 are optically counted independently. Needle loading isoptically checked for total number of loaded items and, further, aradiation detector array scans each needles 19 to confirm that theseed/spacer loading radiation pattern matches the pre-plan. Thisautomatic method and system will do so in the operating room in minimaltime, without the risk of human error in the loading of needles. Theseed loading method will include a pair of redundant 8051microcontrollers (the microprocessors 93 and 94) which will beinterfaced to the dose-planning and implant system computer 16 via aserial port. This interface will read the dose pre-plan information fromthe computer 16, without the need for paper printouts and manualloading. That information will be transferred to a controller whichcontrols the loading of each needle 19. The requirements and designcriteria for the automatic seed-needle loading method and system aredescribed as follows: self-contained and capable of loading seeds andspacers; system will protect operator of system from radiation; dualredundant counting of seeds and spacers; dual redundant radiationdetectors for measuring radiation from active seeds versus spacers; dualredundant measurement of radiation seed positions in needles; systemcheck for failure of either or both redundant counting and measurementsystems; alarm to both operator and to dose-planning and implantcomputer system in the event of error; ongoing account of seed andspacer inventory; tracks needle loading configuration and displays tooperator the designated template grid hole coordinates for each needleloaded; sterilized cassettes for holding seeds and spacers, plussterilizable needle connector; includes one cassette for seeds and onecassette for spacers; dispenses one seed and one spacer at a time, andverifies optically and by radiation detector; system displays needlenumber and template grid location during loading procedure; automaticacquisition of needle loading plan from main system computer; serialinterface with handshake protocol and verification; self-contained(mechanical, power, logic, microcontrollers); operates only if connectedto main system computer.

A convenient storage system for the needles 18 can be loaded by theautomatic seed/needle loading method system. The face of this unit has ahole grid pattern which matches the implant template 25. Loaded needlesmay be inserted into this unit until they are used. The entire unit isshielded for radiation leakage minimization. The template-like face ofthe unit is available in both a reusable, sterilizable version anddisposable versions which match all standard implant template faces.Faces of the unit detach easily and quickly for sterilization ordisposal.

The dose as a function of position for a cylindrical ¹²⁵ I seed of agiven activity can be determined from a lookup table or calculated froma conventional analytic formula. The dose field can be visualized as aset of isodose lines in two-dimensional or isodose surface inthree-dimensional. The dose computation routine is based upon the TG43standard adopted by the AAPM (American Association of Physicists inMedicine) entitled "Dosimetry of Interstitial Brachytherapy Sources":Recommendations of the AAPM Radiation Therapy Committee Task Group No.43 which specifies the dose model and the data used in the dosecalculation. This particular implementation runs extremely fast on aconventional 233 MHz PC, computing the dose for a single seed in lessthan 0.5 seconds. The total three-dimensional dose distribution withinthe prostate for a 100 seed implant requires only 50 seconds, or lessthan one minute total computation time. Thus, this can be done "on line"in the operating room.

In the two-dimensional, three-dimensional perspective, or the real timeVR modes, the user has the ability to view the optimized seeds 18 andthe needles 19 in the same volume as the real time ultrasound data. Thisallows the physician to see exactly where the needles 19 should go andhence make adjustments to position the needles 19 optimally. Thepre-planned, optimal positioned needles 19 and the seeds 18 can berendered again as a transparent solid, the color of which is userselectable. As the real needles 19 are inserted into the prostate, theirpositions relative to the ideal needle placements based on the dose plancan be monitored in real time. Any deviation of the position of a givenneedles 19 can be quickly and accurately readjusted so as to follow thepath of the ideal needles 19. As the different needles 19 are placed atdifferent positions inside the prostate, the viewing angle can beadjusted to facilitate viewing of the needle or catheter placement.FIGS. 5A and 5B displays perspective three-dimensional views and thethree orthogonal reconstructions of the image data along with thepre-planned catheter positions. The pre-planned needles 19 can also beviewed in the VR mode as virtual objects overlaid onto the imagingvolume.

A flowchart description of the translucent volume visualizationmethodology is shown in FIG. 10. The input image volume is described bythe vectors i, j, k of appropriate magnitude for the volume. The viewingangle parameters are the angles θ, .O slashed. described on FIG. 6 andFIG. 10. The rotation matrix, R, is calculated using the formulae givenin the flowchart of FIG. 10. The entire imaging volume is calculated bymultiplying the rotation matrices in the x, y, z directions by therespective vectors i, j and k describing the incremental portions alongthe x, y, z directions. Thus, the multiplying vector is (i-i₀, j-j₀,k-k₀) where i₀, j₀, k₀ are the starting points along x, y and z axes andthe volume is determined by summing the component contributions shown inFIG. 10. The three-dimensional translucent image is then created bycomputing the translucent two-dimensional image over the entire imagevolume and summing the z-pixels.

A virtual reality interface system can be composed of a conventionalhead mounted display (HMD) 50 shown in FIG. 11 and a 6D (x,y,z, roll,pitch, yaw) tracking system. The HMD 50 consists of two color monitorswhich mount to a head set in the position directly in front of the eyes.The HMD 50 is based on the principal that whatever is displayed on eachmonitor is directly incident on the retina for each eye, and hence truethree-dimensional images can be created by rendering objects asthree-dimensional perspective images for each eye. Given the distancebetween the eyes (the interocular distance which is approximately 80 mm)and the distance and spherical angles of the distance of the center linebetween the eyes from the coordinate origin, the two-dimensional imageswhich appear in each of the two monitors can be determined exactly asdescribed above. This results in a true three-dimensional image asperceived by the user. Therefore, as the user moves his or her head ormoves around the room, the distance from the origin and the sphericalangles also change. This motion of the user or user's head can beobtained from the VR tracking system. Given these spatial parameters,the images which are reconstructed in the two eye monitors can beupdated in real time, giving the user the illusion of the object reallyexisting in three-dimensional space. The user literally has the abilityto walk around the object, viewing it in three-dimensional space.

Instead of reconstructing computer generated geometric objects as isusually the case in VR, the transparent, three-dimensionalreconstruction of the real time imaging data will preferably bereconstructed. Hence as the physician walks around the patientundergoing the implant, the physician will see the three-dimensionalultrasound volume mapped inside the patient's pelvis, spatiallycorrelated to the position of the patient's real prostate (or otherorgan) and anatomy. The physician can "see" inside the patient to theextent of what is visible in the ultrasound imaging volume. Since theultrasound probe 12 is locked down to the template, which is thensecured to the floor, the exact positions of all voxels in theultrasound imaging volume are known exactly relative to the template,and hence relative to the room.

As the needles 19 are inserted into the patient, they will appear in theimage volume and hence are reconstructed in the VR reconstruction. Allof this occurs in real time so that the physician also can see theneedles 19 enter the prostate in real time. As mentioned above, if thepre-planned, optimized needles 19 are displayed, the physician can thensee the position of the actual needles 19 as they are being insertedrelative to the optimal placement. Hence, the physician has the abilityto adjust the needles 19 to correspond to their optimal positions. Inaddition, since the needles 19 are automatically extracted, the computersoftware has the ability to calculate and render the three-dimensionaldose distribution in real time as the needles 19 are being inserted.

As an example, a currently available, a fast and inexpensive HMD is madeby Virtual-IO Corporation (Mountain View, Calif.). The HMD is full colorwith two 0.70 LCD displays with a resolution of 180,000 pixels per LCDpanel. The video input is NTSC with field sequential format. The LCDpanels are semitransparent, allowing the real outside world to beincluded in the virtual reconstruction. The field of view is 30° foreach eye. A six degree of freedom (6 DOF) tracking system can also beattached to the HMD. The 6 DOF tracking system allows for thedetermination of the spatial position of the user's head and the yaw,pitch, and roll of the head. The conventional head set weighs only 8ounces and comes with stereo sound. Stereo sound is an extremelyvaluable technology in the operating room. With this capability, thephysician has the ability to monitor the patient's heart rate andrespiration rate while performing the implant. Hence any fluctuation inthe patient's vital signs can be instantly accessed and acted thereon ifnecessary.

The radioactive seeds 18 are made of high density material such asstainless steel, and hence have a very bright response in the ultrasoundimages. Therefore, automatic seed detection in the ultrasound images canreadily be accomplished, for example, by a simple thresholding algorithmalong with the requirement that the resultant objects which are removedby threshold have a certain maximum size determined by the actual sizeof the seeds.

Near-real-time visualization will provide immediate feedback to thephysician during the implant process itself. There is a clear need forthe visualization being available during the implant process. The nearlyreal time visualization is of great importance to the effective use of atranslucent overlay of the ideal seed pre-plan (from the therapyplanning process) in the three-dimensional volume. The physician can"see" in nearly real time the relationship of the needles and seedsbeing implanted to the ideal pre-plan locations and quickly accommodateredirection required prior to leaving the radiation seeds. Further, theneed for this in three-dimensional representation is very important toovercome the greatest fundamental limitation in brachytherapy, which isknowing at the same time both the lateral placement and longitudinalplacement of needles and seeds relative to the target volume andpre-plan. This is a three-dimensional problem which has up until nowbeen addressed in two-dimensional in a stepwise fashion without theability to "see" the exact location of where you are in the target. Thisreal time three-dimensional visualization also would speed the implantprocess in the case of brachytherapy as well as make it more accurate.It would also speed other minimally invasive surgical procedures andlocalized tissue ablation procedures (for example, cryosurgery orlocalized selected ablation of diseased liver tissue or local removal ofbreast tissue). These procedures could be accomplished with real timevisualization inside the tissue being treated with greater accuracy inshorter time. This aspect would reduce operating room time and costs tothe patient and health care system.

While preferred embodiments of the inventions have been shown anddescribed, it will be clear to those skilled in the art that variouschanges and modifications can be made without departing from theinvention in its broader aspects as set forth in the claims providedhereinafter.

What is claimed is:
 1. A system for three-dimensional imaging andtreatment of the body of a patient, comprising:means for developing atherapy plan for treatment of an organ of the patient; means for holdingradioactive seeds and inserting the seeds into position in the patient;an optical sensor positioned to monitor loading of the seeds inaccordance with the therapy plan; means for providing image data from atreatment region of the patient's body; means for providing atranslucent volume image of a portion of a patient's body and a separatetranslucent image of the organ of the patient; and means forillustrating via a translucent image of the means for holding the seedsfor placement of the seeds in conjunction with the organ of the patient,thereby enabling three dimensional viewing of said seeds, means forholding the seeds and simultaneously the organ of the patient and theportion of the patient's body.
 2. An apparatus for preparing radiationtherapy components for treatment of the body of a patient, comprising:atleast one insertion device for holding radioactive seeds, said insertiondevice for passage into the patient and positioning said seeds forradiation treatment; and an optical sensor positioned to monitor loadingof each of said radioactive seeds into said insertion device.
 3. Theapparatus as defined in claim 2 wherein said insertion device furtherholds at least one spacer between said radioactive seeds.
 4. Theapparatus as defined in claim 3 further including a microprocessorcoupled to said optical sensor and programmed to monitor input of atleast one spacer and said radioactive seeds.
 5. The apparatus as definedin claim 3 further including a bin for holding said radioactive seedsand a bin for holding a plurality of said spacers.
 6. The apparatus asdefined in claim 5 further including at least two bins for holdingradioactive seeds of different radioactivity levels.
 7. The apparatus asdefined in claim 5 further including a device to determine whether saidinsertion device is engaged both to said bin for input of said seeds andto said bin for input of said spacers.
 8. The apparatus as defined inclaim 7 further including a programmed microprocessor coupled to adevice for automatically loading said seeds and said spacers into saidinsertion device.
 9. The apparatus as defined in claim 8 furtherincluding a monitor for displaying to an operator an identifier of eachsaid insertion device and status of loading said seeds and said spacersin said insertion device.
 10. The apparatus as defined in claim 8wherein said microprocessor comprises at least two independentmicroprocessors for checking operational status of another of saidmicroprocessors.
 11. The apparatus as defined in claim 8 furtherincluding a system computer having dose plan information which can bedownloaded into said programmed microprocessor for automatically loadingsaid seeds and said spacers into said insertion device.
 12. Theapparatus as defined in claim 11 further including a storage system fora plurality of said insertion device and said storage system coupled tosaid microprocessor for automated positioning of a plurality of saidinsertion device for loading with said seeds and said spacers inresponse to said dose plan information.
 13. The apparatus as defined inclaim 2 wherein each of said radioactive seeds comprise a source of atleast one selectable level of radioactivity.
 14. The apparatus asdefined in claim 2 wherein said optical sensor comprises at least twooptical sensing devices.
 15. The apparatus as defined in claim 14wherein at least two optical sensing devices are redundantly disposedalong one entry passageway.
 16. The apparatus as defined in claim 2further including a microprocessor coupled to said optical sensor andprogrammed to monitor input of said radioactive seeds.
 17. The apparatusas defined in claim 2 further including a radiation sensor disposed tomonitor radioactivity level of said radioactive seeds being input intosaid insertion device.
 18. The apparatus as defined in claim 2 furtherincluding means for implanting said at least one insertion device in thebody of the patient.
 19. The apparatus as defined in claim 18 furtherincluding means for displaying at least a two-dimensional image of atleast one said insertion device and said radioactive seeds when disposedin the patient.
 20. The apparatus as defined in claim 19 whereindifferent transparent colors can be assigned to said insertion device,said radioactive seeds and portions of the patient's body.
 21. Theapparatus as defined in claim 18 wherein said means for implantingincludes a holder positioned adjacent the patient, said holder includinga template having openings for receiving said insertion device.
 22. Theapparatus as defined in claim 21 wherein said template comprises adisposable material for a single use.
 23. The apparatus as defined inclaim 18 further including means for performing tissue ablation of thebody of the patient.
 24. The apparatus as defined in claim 18 furtherincluding an ultrasound imaging system for displaying said radioactiveseeds, said imaging system coupled to a computer programmed to detect animage of said seeds using a threshold ultrasound signal analysisprogram.
 25. The apparatus as defined in claim 2 wherein said insertiondevice is selected from the group of an insertion device and a catheter.26. A method for radiation treatment of the body of a patient,comprising the steps of:positioning a holder adjacent the body of apatient; passing at least one insertion device through openings in saidholder and into the body of the patient; and inputting radioactive seedsand spacers into said insertion device responsive to a preplannedtherapeutic radiation plan and in accordance with programmed computercontrols, the step of inputting said seeds and spacers includingselecting a particular number of radioactive seeds of selected radiationstrength interspersed with said spacers to achieve a calculatedradiation dose level at selected portions in the body of the patient.27. The method as defined in claim 26 further including the planningstep of providing a system to allow stepwise movement of an ultrasoundprobe for determining structure of the selected portion of the patientbefore implanting said insertion device in the patient.
 28. The methodas defined in claim 26 further including the step of generatingthree-dimensional images of said insertion device, said seeds and theselected portions of the body of the patient.
 29. The method as definedin claim 28 wherein a different transparent color is assigned to saidinsertion device, said seeds and the selected portions.
 30. The methodas defined in claim 26 wherein said insertion device is inserted intothe holder in a location in accordance with the radiation plan.
 31. Themethod as defined in claim 26 wherein said holder includes a disposablematerial for a single use.